PIXEL BASED IMAGE RENDERING SYSTEM 



FIELD OF THE INVENTION 
The present invention relates generally to digital imaging and, more 
particularly, to a system for rendering an image to a printer or other image 
output device for producing images made up of a large plurality of pixels. 

BACKGROUND 

Ongoing improvements in personal computers allow users to prepare 
documents of increasing complexity and variety. Printer manufacturers are 
increasingly challenged to develop printing systems capable of priinting high 
quality, high resolution heterogeneous docimients. As the term is used herein, 
heterogeneous documents are documents which include more than one of the 
following: images, graphics, and text. Ideally, heterogeneous documents 
printed by printers accurately reproduce what is composed on a high-resolution 
computer monitor. 

As an example of one of the challenges facing the printing industry, 
computer monitors are based on a color regime of red, green and blue pixels 
(RGB) whereas color printers, namely, Inkjet printers, are typically based on a 
color regime of cyan, magenta, yellow and black (CMYK). The RC;B color 
components of computer moiutors are combined together in an "additive" way 
by mbcing red, green and blue light rays to form a plurality of diff<?rent 
displayable colors. In contrast, the CMYK components of color inks are applied 
to media in different combinations in a "subtractive" way to form ci plxirality of 
printable colors consistent with CMYK ink cartridges typically contained within 
the Inkjet printer. Transforming images from "RGB space" to "CMYK space" 
necessary for printing the image on an Inkjet printer requires the use of color- 
rendering techniques. 
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One color-rendering technique is halftoning. A halftoning algorithm is a 
process of transforming a continuous-tone image into a binary image that gives 
the illusion of the original continuous-tone image. Use of a single halftoning 
algorithm, when rendermg a document, works well when the content of the 
5 document to be printed is homogeneous. For example, if the content of the 
document to be printed consists of only text, or only graphics, etc., an algorithm 
could be selected that will optimally render the text or graphics to the printer. In 
contrast, when a document is heterogeneous, containing various mbctures of text, 
graphics, or images, prior art single halftoning processes do not work as well 

10 because a given halftoning algorithm is directed to optimal rendering of only a 
single type of image. 

Prior art image rendering systems have attempted to address this 
problem. For example, US Patent numbers 5,327,265 and 5,272,549 attempted to 
address the aforementioned problem by using a method wherein both text and 

15 images are printed separately but in the same docimient. These methods include 
combining Inkjet and laser printer components in a printing system in which the 
printed information is divided into color and text. The color images are printed 
using the inkjet printer component and the text (or non-color material) is printed 
by the laser printer component. This approach has the disadvantage of reqmring 

20 very complex printers in order to prmt heterogeneous documents. 

US Patent numbers 5,704,021 and 6,040,927 disclose methods of color 
inkjet printing for use in printing docimients having different color object types. 
Based on the identified color object type, selected color rendering options are 
used by the color inkjet printer to produce the document. The prior art systems 

25 in the '021 and '927 patents require categorizing areas of a heterogeneous 

document to be printed, based on the type of objects contained in regions of the 
document, and then rendering the objects accordingly. 

It would be advantageous to have an image rendering system capable of: 
(1) improving print quality of heterogeneous images, (2) increasing the speed at 
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which heterogeneous images are rendered to a printer and, (3) reridering high 
quality heterogeneous images to relatively inexpensive conventional printers 
such as inkjet printers without being limited by object type. 

SUMMAY OF THE INVENTION 

The present invention provides a pixel based image rendering system and 
method of processing source image data for instructing an output device in 
providing an image having a plurality of pixels. Each pixel has a p>redetermined 
number of data bits which are processed by coupling image information to each 
pixel, assessing the image information for each pixel, and rendering each pixel of 
the image according to the image information coupled to each pixel. 

The invention further provides a pixel based image rendering system for 
instructing a printer in providing an image having a plurality of pixel data sets. 
The image rendering system includes a data storage device for temporarily 
storing the pixel data sets of the image and a software means for associating one 
or more image information bits to each pbcel data set in the data storage device. 
Additionally, the image rendering system contains a print processing means 
responsive to the pixel data sets and the image information bits. The image 
information bits are associated with each pixel data set for rendering the pixels of 
the image using an image rendering process selected by the image information 
associated with each pixel data set. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Figure 1 is a schematic illustration of a prior art printing system used to 

generate and print documents or images. 

Figure 2 is a schematic representation of an image rendering system for 

pixel based image rendering in accordance with the present invention. 

Figure 3(a) is a pictorial representation of a pixel data set of the type used 

as source image data (prior art). 
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Figure 3(b) is a pictorial representation of the pixel data set, as in Figure 
3(a), with the addition of image information bits adjacent a most significant bit of 
the pixel data set. 

Figure 3(c) is a pictorial representation of the pixel data set, as in Figure 
5 3(a), with the addition of image information bits adjacent a least significant bit of 
the pixel data set. 

Figure 3(d) together with the detailed description provided below, 
illustrates how a pixel, as shown in Figure 3(b), is rendered in accordance with 
the present invention. 
10 Figure 4 is a schematic illustration of a sequence of steps by which image 

information is associated with each pixel of the data set. 

Figure 5(a) shows an embodiment of the invention where the image 
rendering system is disposed in a computer. 

Figure 5(b) shows an embodiment of the invention where tlie image 
15 rendering system is disposed a printer. 

Figure 5(c) shows an embodiment of the invention where the image 
rendering system is disposed in a printer controller. 



20 DETAILED DESCRIPTION OF THE EMBODIMENT 

Figure 1 is a schematic representation of a prior art printing system 10 
used to generate and print documents or images. Printing system 10 includes a 
computer 12 having a memory allocation block 14, a graphics device interface 
(GDI) 16, and a print driver 18. The computer 12 is connected to printer 22 or a 

25 plurality of printers via a printing network 20. Box 20 is also referred to herein as 
a network controller or a printing network controller. Printing network 20 is 
connected to a plurality of printers, represented in Figure 1 by printer 22 and 
additional printers 23 and 25, which may be selectively connected to computer 12 
via controller 20. Alternatively, one or more printers may be operatively 
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connected directly to the print driver 18 of computer 12, as illustrated 
schematically in Figure 1 by dashed line 27. In the description of the present 
invention provided below it will be imderstood by those skilled in the art that a 
printer shown connected to a print driver may, alternatively, within the scope of 
the present invention, be connected via a suitable network or via direct 
connection. 

Memory allocation block 14 of computer 12 supports application software 
such as word processors, spreadsheets, and database rendering systems. 
Additionally, computer-aided design and imaging software such as AutoCAD 
and PhotoShop, respectively, are supported by memory allocatior block 14. The 
various software applications may be used to generate homogene(5us or 
heterogeneous images consisting of a multiplicity of pixels. When the pixels of 
the homogeneous or heterogeneous images are to be printed, they form an image 
data stream 24 as they pass through computer 12 in route to printer 22. 

GDI 16 of computer 12 is used by most Windows applications to display 
images on a monitor. The GDI is instrumental when printing from a Windows 
application to a printer because the data (including image data stream 24) 
generated by the program does not need to be converted to printer controller 
language, for example, PostScript (Trademark). Hence, the user Ccin print a 
homogeneous or heterogeneous image, viewable on a computer's monitor, 
directly to a printer, or indirectly to a printer through printing network 20, via 
print driver 18. Print driver 18 converts the application data recei^^ed from GDI 
14 to image data printable on printer 22. 

Figure 2 is a schematic representation of one embodiment of the Image 
Rendering System (IRS) 26 employing an Ulustrative embodiment of the present 
invention. For the purpose of the present invention, IRS 26 refers io a digital 
imaging process for converting image data in its original form, as it emerges 
from an image source such as a GDI, to another form suitable for printing on a 
binary printer such as, for example, an inkjet printer. IRS 26 employes image 
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rendering processes including halftoning processes and color rendering 
algorithms. IRS 26 includes GDI 16, and an IRS print driver 19. IRS print driver 
19 further includes an interim data storage block 28, a coupled image 
information block 30, and a document rendering block 32. 
5 A heterogeneous image 34 created by a software applicatio a may contain 

any combined multiplicity of monochrome or color text pixels, monochrome or 
color image pixels, monochrome or color graphics pbcels, or white pixels. A 
homogeneous image, in contrast, is composed of only one pixel type. A 
homogeneous or heterogeneous image is alternatively referred to herein as a 

10 homogeneous or heterogeneous document. 

GDI 16 operates on the source image pixels and outputs the pixels as an 
image data stream 24 to interim data storage block 28 within IRS p»rint driver 19. 
Blocks 28, 30, and 32 illustrate process steps carried out in IRS print driver 19. 
Within IRS print driver 19, pixel data, which is temporarily stored in interim 

15 storage 28, is coupled to image information, such as information identifying the 
pixel type, in step 30. Attaching image information may sometimes be referred 
to herein as 'adding' image information to each pixel. Adding, for the purpose of 
this invention means associating or coupling image information to each pixel. 
The attached image information is associated with one of four different pbcel 

20 types, as will be described in greater detail below. The modified pixel 

information 36 is then rendered in block 32. IRS print driver 19 renders images 
in accordance with the present invention using attached image information, as 
will be described in below. 

Printer 22, for example, an inkjet printer, then prints a hard copy 38 of 

25 heterogeneous image 34. One skilled in the art will recognize that although IRS 
26 is coupled to only one printer, IRS print driver 19 could be used to render a 
heterogeneous or homogeneous image to a plurality of printers through a 
printing network as illustrate in Figure 1. 
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Figure 3(a) is a pictorial representation of a pixel 40, also referred to as a 
data set or pixel data set, taken from the multiplicity of pixels forming image 
data stream 24 as shown in Figure 2. Pixel 40 as discussed herein is a single point 
in a graphic image capable of being displayed on a computer monitor. The 
number of bits in pixel 40 determines how many variables define the pixel, such 
as colors, intensities or shades of gray, etc. that can be displayed. In the 
illustrated embodiment of the invention, a color monitor or display system uses 
24 bits (8 bits-red; 8 bits-green; 8 bits-blue) for each pixel 40, making it possible to 
display 2 to the 8th power (256) different colors or shades of gray. Each pixel is 
composed of three segments or bytes; red 42, green 44, and blue 4(3 (RGB). Each 
segment of pixel 40 consumes eight bits, shown in Figure 3(a) at 48, or 
equivalently a byte of memory. Furthermore, pixel 40 has a most significant bit 
50 and a least significant bit 52. Pixel 40, when this term is used herein as a data 
set, may alternatively be called a pixel data set. 

Figure 3(b) is a pictorial representation of pixel 40 as shown in Figure 3(a), 
with eight additional image information bits added to pixel 40. As discussed 
herein, a bit is the smallest unit of information on a computing machine such as, 
for example, a computer. A single bit can hold only one of two values: 0 or 1. 
More information is obtained by combining consecutive bits into larger units. For 
example, a byte is composed of 8 consecutive bits. The furthest right bit in a byte, 
as depicted in Figure 3(b), represents the least significant bit and the furthest left 
bit of a byte represents the most significant bit. 

The additional information, henceforth referred to as image information 
54, (or alternatively image information bits) is illustrated as 'ii' in l^^igure 3(b) and 
is added to pixel 40 using a suitable software routine. In the embodiment of the 
invention shown in Figure 3(b), image information 54 is added adjacent the most 
significant bit 50 of pixel data set 40. Alternatively, Figure 3(c) shows 
information bits 54 added adjacent the least significant bit 52 of pixel data set 40. 
Those skilled in the art will recognize that although image information 54 is 
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added to most significant bit 50 or least significant bit 52, image information 54 
may be added to pixel 40 in other positional locations and still fall within the 
scope of the present invention. 

In the image processing system of the present invention the: associating of 
one or more image information bits 54 with each pixel data set 40 Is preferably 
carried out in IRS print driver 19 (Figure 2). The pixel data sets arriving via 
image data stream 24 from GDI 16 are stored in a suitable interim data storage 
memory 28, such as a buffer. The step of coupling or adding image information 
bits 54 to each pixel data set 40 is preferably carried out in memory 28, wherein 
suitable software associates the image information with each pixel . The output 
of the GDI 16 contains the necessary information to display the content of a 
document, be it text, vector, or bitmap, etc. The information is accessible in the 
form of commands and in the data structure of the GDI output. The information 
bits coupled to each pixel in the present invention are derived from the 
commands and data structure of the GDI output. After image information bits 
54 are coupled to each pixel data set 40, the result, termed modified image 
information 36, provides enhanced pixel data which is used in rendering images 
to a printer 22, the resultant images closely approaching the source image 34 
which appeared on the computer monitor. 

In a first embodiment of the present invention, image information 54 is 
coupled to individual pixel data sets to differentiate between four image 
rendering options. If, for example, four image rendering options are available, 
only two image information bits 54 are required for each pixel data set 40. The 
number and types of image rendering options available is a matter of design 
choice and may be as few as two different options, or more that four, depending 
on the number of image information bits 54 associated with each pixel data set 
40. The image rendering options include, for example, selected halftoning 
systems and other image rendering processes used to sharpen, define, or enhance 
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images rendered to printers. Table 1 below provides an illustration of four pixel 
rendering options and associated image information bits 54. 

Table 1: Image information association 



l^t image information bit 


2'^'^ image information bit 


Pixel Rendering 
0]3tions 


0 


0 


V^ite 


0 


1 


Monochrome 


1 


0 


Color image 


1 


1 


Text/edge 



Table 1 and Fig. 3(d) illustrates how the attached image information 54 
affects the rendering of pixel 40. Pixel 40 includes the following 24 bits: 
111111110000000011111110. Information bits 54 are 00000010. The resultant 
modified pixel information 36 becomes: 00000010111111110000000011111110. In 
this example, the image information bits 54 are "10" which corresponds to the 
color image rendering option in Table 1. When modified pixel inf ormation 36 is 
received by image rendering block 32 in IRS print driver 19, as shown in Figure 
2, pixel 40 will be rendered as a color image pixel. As the term is used in the 
present invention, rendering is a manner of taking a continuous tone image and 
replicating it for both color and spatial content using a non-continuous tone 
imaging method. Rendering options relied upon in replicating the continuous 
tone image may include halftoning techniques such as error diffusion and matrix 
thresholding. Employing such rendering options serve to ultimately improve the 
appearance of the printed image. 

In the illustrative embodiment of the present invention, a halftoning 
process is associated with each state of image information 54. Asstiming two 
image information bits, a pixel with a first image information bit of "1" and a 
second image information bit of "1" (11) is rendered, for example, using a 
halftoning process specifically designed to optimally render a text/edge pixel. In 
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contrast, when the first image information bit is a "0" and the second image 
information bit is a "1" (01), the pixel is rendered using a halftoiur.g process 
specifically designed to optimally process a monochrome pixel. A detailed 
description expounding on a method by which each pixel of a hete;rogeneous or 

5 homogeneous image is rendered is described below. 

Figure 4 is a schematic representation of a sequence of stepj! by which 
modified pixel information 36 is assessed and rendered to a printer 22 by IRS 
print driver 19 previously described in Figure 2. The modified pixel information 
36 results from the coupling of two image information bits 54 to pixel data set 40. 

10 Figure 4 is separated into an assessment section 58 and a rendering section 60 as 
described below. 

Assessment section 58 of Figure 4 has a first bit assessment block 62 and 
two second bit assessment blocks, 64 and 66, respectively. First bil: assessment 
block 62 assesses a first image information bit 53 of image information 54 

15 whereas second bit assessment blocks 64 and 66 assess a second image 

information bit 55 of image information 54 [the two rightmost bits (53 and 55) in 
image information blocks 54 of Figure 3(d)]. First bit assessment block 62 
receives each pixel 40 of modified pixel information 36 as shown in Figure 3(d). 
Next, first image information bit 53 of image information 54 is asst!ssed at first bit 

20 assessment block 62. If first image information bit 53 of image information 54 is 
a "1", pixel 40 is channeled to second bit assessment block 64. In contrast, if first 
image information bit 53 of image information 54 is a "0", pixel 40 is channeled 
to second bit assessment block 66. Next, an assessment of second image 
information bit 55 of image information 54 is made at second bit assessment 

25 blocks 64 and 66 respectively. 

In second bit assessment block 64 of Figure 4, if second image information 
bit 55 of image information 54 is a "0" pixel 40 is channeled to pixel rendering 
option 70 for color image pixels. As such, that pixel will be rendered 72 using a 
selected image rendering process suitable for color images, for example, color 
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image halftoning. In contrast, if second image information bit 55 of image 
information 54 is a "1", pixel 40 is channeled to rendering section 75 where pixel 
40 is color converted and subsequently halftoned using a halftoning process 
specifically designed and optimized for a text/ edge pixel. 

Next, consider second bit assessment block 66. If second iraage 
information bit 55 of image information 54 is a ''V pixel 40 is channeled to 
rendering section 80 where pixel 40 imdergoes TRC adjustment and is 
subsequently halftoned using a halftoning process specifically designed and 
optimized for a monochrome pixel. In contrast, if the second image information 
bit 55 of image information 54 is a ''0'', (see second bit assessment block 66) pixel 
40 is channeled to pixel rendering section 85 where pixel 40 does not undergo a 
halftoning process, hence the halftoning process is avoided. 

The location of IRS print driver 19 in IRS 26 shown in Figure 2 is flexible 
as illustrated in Figure 5(a), 5(b), and 5(c), collectively Figure 5. Figiire 5(a) 
shows a preferred embodiment wherein the IRS 26 of the invention is disposed 
in a computer 12. The IRS 26 of the present invention could, howeiver, be 
disposed in a printer 68, as shown in Figure 5 (b). Disposing the invention in 
printer 68 may be accompHshed through implementing the process described in 
Figure 4 within the printer's firmware or hardware (e.g., application specific 
integrated circuits). Another embodiment of the invention is shown in Figure 
5(c). Here, IRS 26 of the invention is disposed in a printer controller 90 located 
external to computer 12 and printer 22. 

From the detailed description, it will be appreciated that the present 
invention provides a pixel based image rendering system having several objects 
and advantages including but not limited to the following: (1) The quality of the 
printed images is improved because each pixel of a homogeneous or 
heterogeneous image is rendered individually according to image information 
attached to each pixel; (2) The speed at which an image is printed is increased 
because some selected pixels are not rendered; and (3) heterogeneous images can 



PSLA 1009 



Page 11 of 17 



be printed on relatively simple, inexpensive printers, such as inkjet printers, 
without being limited by object type. 

Alternative embodiments will become apparent to those skilled in the art 
to which the present invention pertains without departing from its spirit and 
scope. For example the preferred embodiment described herein categorizes each 
pixel into one of four image categories, although the pixels could be 
characterized in more or less categories. A larger number of categories may 
further increase the quality of images printed. 
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